The python computer vision framework.

Construction of image retrieval systems focused on user knowledge interaction.

Learning Directional Local Pairwise Bases with Sparse Coding.

An efficient method for face retrieval from large video datasets.

Building Compact Local Pairwise Codebook with Joint Feature Space Clustering.

Building Compact Local Pairwise Codebook with Joint Feature Space Clustering.

Video Retrieval Based on Tracked Features Quantization.

Unified Approach to Detection and Identification of Commercial Films by Temporal Occurrence Pattern.

PageRank with Text Similarity and Video Near%2DDuplicate Constraints for News Story Re%2Dranking.

Real Time Tunnel Based Video Summarization Using Direct Shift Collision Detection.

High%2Dlevel feature extraction using SVM with walk%2Dbased graph kernel.

Personalized News Video Recommendation.

News Topic Tracking and Re%2Dranking with Query Expansion Based on Near%2DDuplicate Detection.

A Novel Retake Detection Using LCS and SIFT Algorithm.

Editorial.
calls &amp; calendarEDITORIAL
Efficient concept detection by fusing simple visual features.

Editorial.
calls &amp; calendarEDITORIAL
Advances in Multimedia Modeling
Abstract. Attention is a psychological measurement of human reflection against stimulus. We propose a general framework of highlight detection by comparing attention intensity during the watching of sports videos. Three steps are involved: adaptive selection on salient features, unified attention estimation and highlight identification. Adaptive selection computes feature correlation to decide an optimal set of salient features. Unified estimation combines these features by the technique of multi-resolution auto-regressive (MAR) and thus creates a temporal curve of attention intensity. We rank the intensity of attention to discriminate boundaries of highlights. Such a framework alleviates semantic uncertainty around sport highlights and leads to an efficient and effective highlight detection. The advantages are as follows: (1) the capability of using data at coarse temporal resolutions; (2) the robustness against noise caused by modality asynchronism, perception uncertainty and feature mismatch; (3) the employment of Markovian constrains on content presentation, and (4) multi-resolution estimation on attention intensity, which enables the precise allocation of event boundaries. Key words:highlight detection, attention computation, sports video analysis 1
Scene duplicate detection based on the pattern of discontinuities in feature point trajectories.

Image%2Dbased quiz generation from news video archives based on principal object.

Unsupervised Face Annotation by Mining the Web.

New Approach for Hierarchical Classifier Training and Multi%2Dlevel Image Annotation.

A Novel Approach for Filtering Junk Images from Google Search Results.

New Approach for Hierarchical Classifier Training and Multi%2Dlevel Image Annotation.

A text segmentation based approach to video shot boundary detection.

ISM TRECVID2008 High%2Dlevel Feature Extraction.
We studied a method using support vector machines (SVMs) with walk-based graph kernels for the high-level feature extraction (HLF) task. In this method, each image is first segmented into a finite set of homogeneous segments and then represented as a segmentation graph where each vertex is a segment and edges connect adjacent segments. Given a set of features associated with each segment, we then obtain a positive definite kernel between images by comparing walks in the respective segmentation graphs, and image classification is carried out with an SVM based on this kernel. We submitted six runs using this method with several combinations of the values of the kernel and SVM parameters. 1.
National Institute of Informatics%2C Japan at TRECVID 2008.
Abstract—This paper reports our experiments for TRECVID 2008 tasks: high level feature extraction, search and contentbased copy detection. For the high level feature extraction task, we use the baseline features such as color moments, edge orientation histogram and local binary patterns with SVM classifiers and nearest neighbor classifiers. For the search task, we use different approaches including search by the baseline features and search by concept suggestion. And for the video copy detection task, we study two approaches that are based on the pattern of motions in feature point trajectories and matches of all frame pairs using normalized cross correlation. Our approaches can be considered as one of the baseline approaches for evaluation of these tasks. I. HIGH LEVEL FEATURE EXTRACTION A. Method Overview

Rushes summarization using different redundancy elimination approaches.


Media objects for user%2Dcentered similarity matching.

mediaWalker a video archive explorer based on time%2Dseries semantic structure.

Video search by multi%2Dmodal and clustering analysis.


Using Visual%2DTextual Mutual Information and Entropy for Inter%2Dmodal Document Indexing.


Image%2DBased Quizzes from News Video Archives.

Boosting Face Retrieval by using Relevant Set Correlation Clustering.

Scene duplicate detection from videos based on trajectories of feature points.

Large scale news video database browsing and retrieval via information visualization.



National institute of informatics

Mining Large%2DScale News Video Database Via Knowledge Visualization.
Polytechnic University
Ent%2DBoost Boosting using entropy measures for robust object detection.

Large%2Dscale news video retrieval via visualization.

Using Topic Concepts for Semantic Video Shots Classification.
Abstract. Automatic semantic classification of video databases is very useful for users searching and browsing but it is a very challenging research problem as well. Combination of visual and text modalities is one of the key issues to bridge the semantic gap between signal and semantic. In this paper, we propose to enhance the classification of highlevel concepts using intermediate topic concepts and study various fusion strategies to combine topic concepts with visual features in order to outperform unimodal classifiers. We have conducted several experiments on the TRECVID’05 collection and show here that several intermediate topic classifiers can bridge parts of the semantic gap and help to detect high-level concepts. 1
Face Retrieval in Broadcasting News Video by Fusing Temporal and Intensity Information.

Robust Object Detection using Fast Feature Selection from Huge Feature Sets.

Exploiting Topic Thread Structures in a News Video Archive for the Semi%2DAutomatic Generation of Video Summaries.

Identification and Detection of the Same Scene Based on Flash Light Patterns.

Ent%2DBoost Boosting Using Entropy Measure for Robust Object Detection.

Analyzing Person Information in News Video.

A hybrid classifier for precise and robust eye detection.

Analyzing Person Information in News Video.

A Multi%2DStage Approach to Fast Face Detection.
This paper describes a multi-stage approach for achieving fast and robust face detection. This approach was motivated by the work of Viola and Jones [7] using a cascade of classifiers yielding a coarse-to-fine strategy to significantly reduce detection time while maintaining high detection rate. However, it is distinguished from the previous work by two facts: (i) First, a new stage is added to more quickly estimate face candidate regions by using a larger window size and a larger moving step size. (ii) Second, we propose using SVM classifiers instead of AdaBoost classifiers in the last stage and study how to efficiently reuse Haar wavelet features selected by AdaBoost in the previous stage for SVM classifiers. By combining AdaBoost and SVM classifiers, the final system can obtain both fast and robust detection because most of the non-face patterns are quickly rejected in earlier layers, while only a small number of promising face patterns are robustly classified in the later layers. Extensive experimental results demonstrated that our proposed system can achieve promising results. 1
Cooking navi assistant for daily cooking in kitchen.

trackThem Exploring a Large%2DScale News Video Archive by Tracking Human Relations.

An Efficient Feature Selection Method for Object Detection.
Abstract. We propose a simple yet efficient feature-selection method — based on principle component analysis (PCA)  — for SVM-based classifiers. The idea is to select features whose corresponding axes are closest to the principle components computed from a data distribution by PCA. Experimental results show that our proposed method reduces dimensionality similar to PCA, but maintains the original measurement meanings while decreasing the computation time significantly. 1
Examination and enhancement of a ring%2Dstructured graphical search interface based on usability testing.

Advances in Multimedia Information Processing %2D PCM 2004

Advances in Multimedia Information Processing %2D PCM 2004

Advances in Multimedia Information Processing %2D PCM 2004

Topic Threading for Structuring a Large%2DScale News Video Archive.


Generalized Histogram Empirical Optimization of Low Dimensional Features for Image Matching.


Key Image Extraction from a News Video Archive for Visualizing Its Semantic Structure.

A News Video Browser Using Identical Video Segment Detection.

Mining Large%2DScale Broadcast Video Archives Towards Inter%2Dvideo Structuring.

Multimedia Integration for Cooking Video Indexing.

Distinctiveness%2DSensitive Nearest Neighbor Search for Efficient Similarity Retrieval of Multimedia Information.
Nearest neighbor (NN) search in high dimensional feature space is widely used for similarity retrieval of multimedia information. However, recent research results in the database literature reveal that a curious problem happens in high dimensional space. Since high dimensional space has high degree of freedom, points could be so scattered that every distance between them might yield no significant difference. In this case, we can say that the NN is indistinctive because many points exist at the similar distance. To make matters worse, indistinctive NNs require more search cost because search completes only after choosing the NN from plenty of strong candidates. In order to circumvent the harmful effect of indistinctive NNs, this paper presents a new NN search algorithm which determines the distinctiveness of the NN during search operation. This enables us not only to cut down search cost but also to distinguish distinctive NNs from indistinctive ones. These advantages are especially beneficial to interactive retrieval systems.
Semantic Retrieval in a Large%2DScale Video Database by Using Both Image and Text Feature.

Subject region segmentation in disparity maps for image retrieval.

Distinctiveness%2DSensitive Nearest Neighbor Search for Efficient Similarity Retrieval of Multimedia Information.
Nearest neighbor (NN) search in high dimensional feature space is widely used for similarity retrieval of multimedia information. However, recent research results in the database literature reveal that a curious problem happens in high dimensional space. Since high dimensional space has high degree of freedom, points could be so scattered that every distance between them might yield no significant difference. In this case, we can say that the NN is indistinctive because many points exist at the similar distance. To make matters worse, indistinctive NNs require more search cost because search completes only after choosing the NN from plenty of strong candidates. In order to circumvent the harmful effect of indistinctive NNs, this paper presents a new NN search algorithm which determines the distinctiveness of the NN during search operation. This enables us not only to cut down search cost but also to distinguish distinctive NNs from indistinctive ones. These advantages are especially beneficial to interactive retrieval systems.
Subject region segmentation in disparity maps for image retrieval.

Comparative Evaluation of Face Sequence Matching for Content%2DBased Video Access.
The paper presents comparative evaluation of matching methods of face sequences obtained from actual videos. Face information is quite important in videos, especially in news programs, dramas, and movies. Accurate face sequence matching enables many multimedia applications including content-based face retrieval, automated face annotation, video authoring, etc. However, face sequences in videos are subject to variation in lighting condition, pose, facial expression, etc., which cause difficulty in face matching. In order to cope with this problem, several face sequence matching methods are proposed by extending face still image matching, traditional pattern recognition, and recent pattern recognition techniques. They are expected to be applicable to face sequences extracted from actual videos. The performance of these methods are evaluated as the accuracy of face sequence annotation using the methods. The accuracy is evaluated using considerable amount of actual drama videos. The evalua...
an actor%2Factress annotation system for drama videos.

Face Sequence Matching with Certainty Factor Evaluation.
The utilization of face information enables many multimedia applications, especially for news pro-grams, dramas, and movies. In order to utilize face information, face matching methods play important role. But, face matching is difficult, because face information has variation according to facial expres-sion, pose, lighting condition, etc. In order to cope with this problem, we introduce a rejection mecha-nism in face sequence matching. It is expected that faces with not enough information for matching are excepted by the rejection mechanism taking advan-tage of certainty factor evaluation. This paper de-scribes a method of robust face sequence matching with rejection mechanism. 1
Toward the MEdiaSys VIdeo Search Engine %28MEVISE%29.

Application of multidimensional indexing methods to massive processing of multimedia information.

an actor%2Factress annotation system for drama videos.

Towards actor%2Factress identification in drama videos.

Toward Multimedia Document Support inside the AHYDS Platform.

An Efficient Implementation and Evaluation of Robust Face Sequence Matching.

Name%2DIt Naming and Detecting Faces in News Videos.
ions. (In the near future, the worldwide trend will be for broadcasts to feature closed captions.) Thus we use closed-caption texts as transcripts for news videos. In addition, we employ video-caption detection and recognition. We used &#034;CNN Headline News&#034; as our primary source of news for our experiments. Given image sequences, transcripts, and video captions as information sources, Name-It associates extracted faces with extracted name candidates using the correlation of their timing information and face similarity information. Video captions are also taken into account as supplementary information. To associate faces and names, Name-It integrates several advanced image processing and natural-language processing techniques ---face sequence extraction and similarity evaluation from videos, name extraction from transcripts, and video-caption recognition. Although these technologies aren&#039;t always highly accurate, integrating these results will help the system achieve more accurate output
Video OCR Indexing Digital News Libraries by Recognition of Superimposed Captions.
. The automatic extraction and recognition of news captions and annotations can be of great help locating topics of interest in digital news video libraries. To achieve this goal, we present a technique, called Video OCR (Optical Character Reader), which detects, extracts, and reads text areas in digital video data. In this paper, we address problems, describe the method by which Video OCR operates, and suggest applications for its use in digital news archives. To solve two problems of character recognition for videos, low resolution characters and extremely complex backgrounds, we apply an interpolation filter, multi-frame integration and character extraction filters. Character segmentation is performed by a recognition-based segmentation method, and intermediate character recognition results are used to improve the segmentation. We also include a method for locating text areas using text-like properties and the use of a language-based postprocessing technique to increase word recogni...
SR%2Dtree An index structure for nearest%2Dneighbor searching of high%2Ddimensional point data.

Name%2DIt Association of Face and Name in Video.
This paper proposes a novel approach to extract meaningful content information from video by collaborative integration of image understanding and natural language processing. As an actual example, we developed a system that associates faces and names in videos, called Name-It, which is given news videos as a knowledge source, then automatically extracts face and name association as content information. The system can infer the name of a given unknown face image, or guess faces which are likely to have the name given to the system. This paper explains the method with several successful matching results which reveal effectiveness in integrating heterogeneous techniques as well as the importance of real content information extraction from video, especially face-name association. This material is based upon work supported by the National Science Foundation under Cooperative Agreement No. IRI-9411299. Any opinions, findings, and conclusions or recommendations expressed in this material are ...
Name%2DIt Naming and Detecting Faces in Video by the Integration of Image and Natural Language Processing.
We have been developing Name-It, a system that associates faces and names in news videos. First, as the only knowledge source, the system is given news videos which include image sequences and transcripts obtained from audio tracks or closed caption texts. The system can then either infer the name of a given face and output the name candidates, or can locate the faces in news videos by a name. To accomplish this task, the system extracts faces from image sequences and names from transcripts, both of which might correspond to key persons in news topics. The proposed system takes full advantage of advanced image and natural language processing. The image processing contributes to the extraction of face sequences which provide rich information for face-name association. The processing also helps to select the best frontal view of a face in a face sequence to enhance the face identification which is required for the processing. On the other hand, the natural language processing effectively...
The SR%2Dtree An Index Structure for High%2DDimensional Nearest Neighbor Queries.
Recently, similarity queries on feature vectors have been widely used to perform content-based retrieval of images. To apply this technique to large databases, it is required to develop multidimensional index structures supporting nearest neighbor queries e ciently. The SS-tree had been proposed for this purpose and is known to outperform other index structures such as the R*-tree and the K-D-B-tree. One of its most important features is that it employs bounding spheres rather than bounding rectangles for the shape of regions. However, we demonstrate in this paper that bounding spheres occupy much larger volume than bounding rectangles with high-dimensional data and that this reduces search efficiency. To overcome this drawback, we propose a new index structure called the SR-tree (Sphere/Rectangle-tree) which integrates bounding spheres and bounding rectangles. A region of the SR-tree is specified by the intersection of a bounding sphere and a bounding rectangle. Incorporating bounding rectangles permits neighborhoods to be partitioned into smaller regions than the SS-tree and improves the disjointness among regions. This enhances the performance on nearest neighbor queries especially for highdimensional and non-uniform data which can be practical in actual image/video similarity indexing. We include the performance test results that verify this advantage of the SR-tree and show that the SR-tree outperforms both the SS-tree and the R*-tree.   
A New Type of Video Scene Classification System Based on Typical Model Database.

An automated generation of an electronic library based on document image understanding.

A rule learning method for academic document image processing.

Robust Line Drawing Understanding Incorporating Efficient Closed Symbols Extraction.
This paper presents an efficient extraction method of closed loops as primitive symbols of line drawings, and an robust line drawing un-derstanding system incorporating this extrac-tion method. A graph search technique is used for the symbols extraction method to provide efficient model prediction and verification fa-cilities, and quantitative consideration on the computational cost is shown. The line draw-ing understanding system can efficiently ex-tract models from fairly complicated line draw-ings preserving the computational cost fairly small. 1
A Syntactical Approach to the Database Construction Method from Document Images.

Understanding Rule Generation Supporting System for Drawing Understanding using Interaction with User.
It is nrcessary 70 artlieve rfftcirnt and powrrT11I drawing irnderstandinpl in orrlcr to r~alize lar~e SCRIP ptactiral drawing image datalmes. Resides, a drawing untlrrslanding syst~n ~ is llard to hirild, so this ran be a bottle-n~ck of walixin ~ I lipsp dat ahil~es. Thus autornaf ic genpratitrt~ of (Irxwrng ~~ndrrstanding system is desirable. tVp I ~ V P prmpos~d a drawing understandir~g ~ly~t~rn using state transition models. This system performs understanding process complying with understanding rules which have declarative form to he handld rahily by tnxrt~inps. Here we prearnt an example of drawing i~nrl~r~tandiilg system g~neration syqtem using man-machine interaction. This system storeq pt~viotts RPSS~C) ~ with user nnd avoids rrpeatink same query t~aing computer learning n~ethod. In this pappr, alter brief introduction of the system is pr~sented. embodiment of an experimental systrm and succ~ssful pxp~rirnrtits are show!i to reveal rtTrctiv~n~ss of the system.
Descriptive Ability of Drawing Image Understanding Framework Using State Transition Models.
It is strongly desired to construct multimedia databases especially of maps or engineering drawings. If these databases are intended to be used really effec-tively, original drawing images should be recognized as completely as possible. We have been proposed a new framework of the drawing image understanding system to achieve this. This framework has following features: a) the understanding knowledge is free from the understanding mechanism, b) the understanding knowledge can be ob-tained relatively easily, c) the understanding mechanism can be used commonly among its applications. Some understanding systems based on this framework have been implemented, and it is found that some kind of drawings can be applied this framework very easily, but some of them are not so easy to apply. In this paper, out-line of this framework is described first. Then applying this framework to some kinds of drawing images, we re-veal that what kind of drawings are applicable and what kind of them are not. And discussion of how to extend our framework to apply this to the latter drawings is pre-sented. 1
